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Sensemaking  in  the  Fog  of  War:  An  Experimental  Study  of  Flow 
Command  Teams  Arrive  at  a  Basis  for  Action 

Abstract 

A  commander  and  his  staff  have  to  use  whatever  information  is  available  to  arrive  at  an 
understanding  of  the  situation  that  can  serve  as  a  basis  for  action,  hence  sensemaking  has  a 
central  role  in  the  C2  process.  This  paper  reports  the  results  of  a  first  experimental  study  of 
the  sensemaking  process  in  a  command  team.  The  participants  were  99  Army  captains 
studying  at  the  Swedish  National  Defence  College.  They  worked  in  16  teams  of  5-7 
individuals,  and  one  team  member  acted  as  brigade  commander.  The  task  was  to  produce  a 
part  of  a  brigade  order  within  six  hours.  All  teams  were  video  taped  for  the  entire  session. 
Eight  teams,  assigned  to  the  low  uncertainty  condition,  received  information  about  the 
location  of  all  enemy  units.  The  remaining  eight  teams,  in  the  control,  “normal”  condition 
received  what  might  normally  be  expected.  It  proved  possible  to  distinguish  two  sensemaking 
processes,  one  concerned  with  understanding  the  mission  and  what  it  required,  and  one  that 
involved  producing  the  basis  for  the  plan.  The  quality  of  the  plan,  as  judged  by  experts,  was 
related  to  the  characteristics  of  the  sensemaking  process,  as  assessed  from  the  video 
recordings. 


Introduction 

’’Sensemaking  is  the  essential  link  to  information  and  decision  superiority, 
but  remains  a  weak  link  in  the  C2  value  chain”  (CCRP  &  AIAA  TC  IC2S, 

2001,  p.  19). 

We  consider  sensemaking  in  command  and  control  the  process  of  achieving  an  understanding 
of  a  situation  in  terms  of  what  to  do.  It  is  a  central  task  in  military  decision  making,  and  in  the 
collective  work  of  a  military  staff. 

Despite  the  obvious  importance  of  sensemaking,  it  has  received  little  study  in  the  context  of 
military  decision  making.  Most  important,  there  seem  to  be  no  experimental  studies  of 
sensemaking  in  C2,  so  we  know  virtually  nothing  about  what  factors  affect  the  quality  of  the 
sense  achieved  by  a  commander  and  his/her  staff.  The  present  paper  is  a  first  report  from  a 
project  aimed  at  filling  this  gap  in  our  understanding.  The  experiment  reported  here  was 
designed  to  measure  the  quality  of  the  sensemaking  process  in  command  teams  involved  in 
military  planning.  The  independent  variable  was  degree  of  uncertainty.  In  one  of  the 
experimental  conditions,  the  teams  planned  in  the  normal  fog  of  war,  i.e.,  with  a  “normal” 
level  of  uncertainty.  In  the  other  condition,  the  fog  had  been  lifted  (presumably  by  electronic 
means)  and  the  planning  took  place  under  conditions  with  near  perfect  information,  as 
envisioned  for  the  new  network  based  defence.  Our  focus  is  on  the  sensemaking  process. 
However,  we  also  assess  the  correspondence  between  ratings  of  the  sensemaking  process  and 
the  quality  of  the  plans  produced  by  the  command  teams. 


Sensemaking  -  Theories  and  Models 

Weick  (1995)  may  be  considered  the  father  of  the  concept  of  sensemaking,  although  the 
concept  clearly  has  a  grandfather  in  James’s  (1890)  theory. 


2 


Sensemaking  is  not  situation  assessment  (e.g.,  Klein,  Orasanu,  Calderwood  &  Zsambok, 
1993;  Klein  et  al.,  2003),  and  sense  is  not  situation  awareness  (e.g.,  Endsley,  1994a,  1994b). 
Nor  can  situation  assessment  and  situation  awareness  be  seen  as  components  of  the 
sensemaking  process.  Sensemaking  has  different  roots  and  represents  a  different  view  of 
human  cognitive  processes  than  situation  awareness  does.  Thus,  situation  awareness  refers  to 
how  detailed  and  accurate  knowledge  an  individual  (or  a  group  of  people)  has  of  a  given 
situation  (Endsley,  1994a,  1994b;  Strater,  Endsley,  Pleban  &  Matthews,  2001),  and  the  extent 
to  which  the  individual  can  predict  what  will  happen  next.  Situation  assessment  assumes  a 
known  situation  for  which  the  result  of  the  assessment  can  be  compared  to  the  true  situation, 
while  a  sense  arrived  at  might  have  little  in  common  with  any  “true”  situation,  for  there  is 
simply  no  way  of  knowing  what  the  true  situation  is,  except  by  acting  on  the  basis  of  the 
sense  that  has  been  achieved.  Life  rarely  offers  the  opportunity  for  people  to  check  if  their 
present  view  of  a  situation  is  wholly  correct.  The  only  information  they  receive  is  whether  the 
actions  resulting  from  the  sense  gained  produce  the  intended  effects.  The  sense  will  be 
considered  correct  if  the  actions  based  on  it  are  followed  by  the  expected  results  (James, 
1907).  Therefore,  measurements  of  sensemaking  should  focus  on  the  quality  of  the 
sensemaking  process  and  the  outcome  of  the  resulting  actions  more  than  on  how  people 
picture  the  situation. 

An  important  aspect  of  Weick's  (1995)  theory  of  sensemaking  that  he  is  perhaps  not  that 
explicit  about  in  his  definition,  but  which  is  still  present,  is  that  sensemaking  generally  take 
place  in  the  service  of  some  goal,  and  that  this  goal  has  precedence.  People  thus  do  not  take 
in  a  situation  “as  such”;  they  study  it  for  a  purpose.  The  goal  could  be  to  find  arguments  to 
convince  others  of  some  position,  to  confirm  prior  expectations,  to  strengthen  ones 
commitment  to  some  decision  taken,  and  so  on.  From  these  examples  it  is  obvious  that 
sensemaking  is  by  no  means  always  undertaken  in  search  of  the  “truth”. 

In  command  and  control,  the  goal  is  generally  to  solve  some  pressing  problem.  When  a 
military  staff  studies  a  situation,  it  does  so  because  it  has  a  mission  to  accomplish,  not 
because  they  are  interested  in  how  things  are.  Different  missions  would  make  the  commander 
and  his/her  staff  focus  on  different  aspects  of  the  situation,  and  make  different  sense  of  it. 
This  is,  of  course,  true  also  of  the  pilots  who  are  the  primary  subjects  in  studies  of  situation 
awareness  (Endsley,  1994a,  1994b;  Strater,  Endsley,  Pleban  &  Matthews,  2001).  They  most 
likely  arrive  at  a  different  sense  of  the  situation  and  the  other  aircraft  around  them  when  they 
are  in  combat  compared  to  when  they  are  just  practicing  flying  in  formation. 

Since  we  have  not  found  any  useful  model  of  sensemaking,  as  we  understand  it,  in  military 
decision  making,  we  have  developed  our  own.  The  “data/frame  model  of  sensemaking” 
proposed  by  Klein  et  al.  (2003)  is  a  model  of  individual,  not  collective,  sensemaking,  and  it 
looks  more  like  a  fairly  traditional  model  of  individual  learning  in  cognitive  psychology. 
Moreover,  it  is  not  possible  to  make  any  predictions  from  the  model  as  to  how  different 
factors  would  influence  sensemaking. 

We  now  turn  to  a  brief  description  of  our  model. 


A  Model  of  Sensemaking  in  Military  Decision  Making 
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Our  model  is  shown  in  Figure  1.  It  is  based  on  Weick’s  (1995;  Weick  &  Sutcliffe,  2001) 
theory.  The  model  considers  the  individual  staff  members  as  well  as  the  staff  as  a  unit,  and 
treats  the  commander  as  an  individual  serving  a  specific  function. 

In  military  staff  work,  each  and  every  staff  member  arrives  at  his  or  her  view  of  what  is 
possible  to  achieve  in  the  situation,  an  individual  sense,  depending  on  his  or  her  function  in 
the  staff,  and  from  the  perspective  implied  by  that  function.  This  takes  place  in  interaction 
with  the  other  staff  members.  Sensemaking  is  never  a  truly  individual  activity.  Information 
and  opinions  are  exchanged,  individuals  influence,  and  are  influenced  by,  the  other  staff 
members.  Together  the  staff  creates  a  more  or  less  clear  and  elaborate  shared  sense.  The 
commander  is  a  special  kind  of  an  individual,  because  it  is  he  or  she  who  ultimately  decides 
on  the  course  of  action,  and  thus  what  sense  is  to  be  made  of  the  situation.  We  call  this  the 
commander  sense. 

The  commander  has  been  assigned  a  mission  and  equipped  with  resources  that  may  be  used 
to  accomplish  it,  as  well  as  certain  restrictions  in  the  form  of  rules  of  engagement.  The  staff 
supports  him  or  her.  For  each  individual  staff  member,  it  is  the  mission  in  combination  with 
the  individual’s  task-related  knowledge,  function  in  the  staff,  and  his  or  her  personal  values 
that  determine  what  information  available  will  be  attended  to.  The  (preliminary)  sense, 
arrived  at  individually  and  as  a  consequence  of  the  collaboration  among  staff  members, 
serves  to  direct  further  search  for  information. 

It  is  assumed  that,  if  the  staff  members  generously  share  their  views  with  each  other,  and 
engage  in  discussion,  this  will  have  favorable  effects  on  the  quality  of  both  shared  and 
individual  sense,  and  will  further  the  quality  of  the  course  of  action  eventually  decided  upon. 

Three  factors  are  assumed  to  have  important  effects  on  this  process: 

1.  The  degree  of  shared  knowledge.  The  more  each  staff  member  has  to  explain  to, 
and  teach,  the  others  to  convey  his  or  her  own  understanding,  the  slower  and  more 
cumbersome  the  process,  because  it  requires  explicit  communication  where 
implicit  understanding  would  have  been  enough  if  the  staff  members  had  a  greater 
store  of  shared  knowledge. 

2.  The  social  climate  and  interaction  in  the  staff.  Is  everybody  listened  to?  Does 
everybody  do  his  or  her  best  to  contribute?  Are  all  views  considered  seriously?  Do 
the  staff  members  feel  safe  to  express  their  opinions  freely?  It  is  assumed  that 
affirmative  answers  to  these  questions  imply  better  sensemaking. 

3.  The  organization  of  work  as  well  as  the  role  taken  by  the  commander,  and  whether 
he  or  she  takes  part  in  the  work,  most  certainly  has  an  impact  on  the  sensemaking 
process.  The  effect  may  be  positive  because  the  commander  generally  has  more 
knowledge  and  experience  than  the  members  of  his/her  staff.  If  the  commander 
does  not  encourage  free  exchange  of  ideas,  and  demands  submission  by  the  staff 
members  to  his  or  her  own  views,  this  is  assumed  to  have  a  negative  effect. 

These  considerations  lead  to  the  model  illustrated  in  Figure  1.  The  commander  sense  will  be 
expressed  in  a  commander’s  intent,  and  orders/missions  to  subordinate  units  will  be  based  on 
this  intent.  This  will  eventually  bring  about  some  effect  in  the  “real  world”,  and  information 
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about  the  result  will  be  fed  back  to  the  staff,  and  perhaps  result  in  new  incoming  orders,  or  a 
new  mission. 


Figure  1.  The  Model  of  Sensemaking  in  Military  Decision  Making 

The  Present  Study 

Earlier  studies  of  sensemaking  have  generally  been  case  studies  (e.g.,  Graeme  &  Brown, 
2003;  Miller,  2003;  Murphy,  2001;  Weick,  2001;  Weick  &  Sutcliffe,  2001).  Klein  et  al. 
2002),  however,  compared  expert  and  novice  sensemaking  performance.  They  designed 
scenarios  that  were  presented  piecemeal  to  the  participants.  The  participants  gradually 
acquired  more  and  more  information  about  the  situation,  and  their  thinking  process  was 
assessed  repetitively  during  the  session.  The  participants  were  asked  to  interpret  the  situation, 
to  describe  major  themes  and  vital  issues  in  the  scenario. 

In  the  present  study,  as  noted  above,  we  assessed  the  sensemaking  process  of  the  command 
teams,  and  compared  the  ratings  of  the  quality  of  the  sensemaking  process  with  the  expert 
ratings  of  the  quality  of  the  plans  developed  by  the  teams.  Our  question  was  if  it  would  be 
possible  to  predict  the  quality  of  the  plans  from  an  evaluation  of  the  sensemaking  process, 
and  whether  the  quality  of  the  sensemaking  was  affected  by  the  quality  of  the  information 
provided.  The  effects  of  the  information  on  the  quality  of  the  plans  is  discussed  elsewhere 
(Thunholm,  2005). 
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Method 


Participants 

Participants  were  16  teams  of  5-7  Army  captains  from  the  Staff  program  (the  program 
required  before  a  captain  is  promoted  to  major)  at  the  Swedish  National  Defence  College,  in 
total  99  individuals  of  whom  two  were  women.  Their  mean  age  was  32  years.  The  exercise, 
designed  as  an  experiment,  was  part  of  their  course  work  at  the  college.  The  participants  had 
been  working  together  in  their  teams  for  the  previous  7  months.  The  teams  were  free  to 
organize  the  work  and  roles  in  their  team,  but  one  individual,  elected  by  the  team  members, 
should  serve  as  brigade  commander,  one  should  be  responsible  for  the  communication  with 
the  superior  head  quarters  and  with  subordinate  units,  and  one  should  document  the  process. 
For  the  present  analysis,  eight  of  the  16  teams  were  selected  for  a  detailed  analysis, 
comprising  the  two  teams  that  produced  the  best  and  the  worst  plans  in  each  experimental 
condition. 

Design 

As  explained  above,  the  independent  variable  was  level  of  uncertainty.  There  were  two  levels 
of  uncertainty,  with  eight  teams  randomly  assigned  to  each  condition.  In  the  low  uncertainty 
condition,  the  command  teams  were  given  the  kind  of  information  that  they  would  be 
expected  to  have  in  the  new  network  based  defence  (network  enabled  defence,  or  network 
centric  warfare,  the  terms  used  differ  from  country  to  country),  i.e.,  full  information  about  the 
location  of  enemy  units.  In  the  normal  uncertainty  condition,  neither  the  exact  location  nor 
the  number  of  the  enemy  units  was  known.  The  task  was  to  produce  a  part  of  a  brigade  order. 

Scenario 

The  scenario  was  a  battle  scenario  at  the  tactical  (brigade)  level.  In  this  scenario  a 
mechanized  division  from  the  fictitious  country  Angripien  (“Attackia”)  has  been  air  dropped 
and  landed  in  the  Swedish  cities  of  Stockholm  (the  capital  of  Sweden)  and  Nynashamn  (a  sea 
port  south  of  Stockholm)  and  they  have  established  a  bridgehead.  The  Swedish  1st 
Mechanized  Division  should  now  be  ready  to  launch  an  attack  with  two  brigades  from  the 
north  to  open  up  communications  and  subsequently  attack  and  defeat  the  enemy. 

Procedure 

Six  weeks  prior  to  the  experiment,  the  participants  were  informed  about  how  the  exercise 
would  be  organised.  They  also  received  a  description  of  the  evolution  of  the  scenario  up  to 
the  point  in  time  when  the  exercise  would  begin. 

The  day  before  the  experiment,  the  teams  received  information  about  the  organization  and 
equipment  of  their  units,  and  maps  of  the  area.  The  teams  were  allowed  two  scheduled  hours 
to  prepare  for  the  exercise. 

On  the  day  of  the  experiment,  the  teams  received  a  short  description  of  their  mission.  They 
also  received  general  instructions  about  what  they  were  expected  to  do  in  addition  to 
producing  an  order,  such  as  documenting  their  planning  activities  and  answering 
questionnaires  at  specified  occasions  (Thunholm,  2005).  They  were  then  allowed  45  minutes 
to  make  preparations  and  test  communication  channels  (e-mail).  After  that,  they  were 
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instructed  to  send  a  team  member  to  collect  the  division  order,  and  that  marked  the  beginning 
of  the  exercise. 

The  teams  had  six  hours  to  produce  their  order.  Initially,  they  were  given  the  mission  for  their 
brigade  as  well  as  information  about  the  enemy  and  about  own  forces.  After  20  minutes,  the 
division  commander  assembled  all  team  commanders  for  a  briefing.  The  commanders  of  the 
teams  planning  with  normal  information  returned  to  their  teams  after  40  minutes,  while  the 
commanders  of  the  teams  receiving  detailed  intelligence  on  enemy  units  remained  an 
additional  15  minutes  before  returning  to  their  teams.  All  teams  could  communicate  with 
their  subordinate  commanders  to  ask  about  the  whereabouts  of  own  units.  These  were, 
however,  under  transport  and  could  be  relied  upon  to  be  at  their  assigned  destinations  when 
so  expected.  The  teams  could  request  additional  information  from  the  superior  commander, 
and  they  would  receive  whatever  was  consistent  with  their  respective  uncertainty  condition. 

Each  team  was  videotaped  for  the  whole  six-hour  session.  The  tapes  were  then  analysed  with 
respect  to  both  the  product  and  the  nature  of  the  sensemaking  process. 

Analysis 

Coding  of  Videos.  The  first  author  analyzed  the  video  recordings,  and  the  actions  of  the  team 
members  were  coded  into  a  number  of  categories,  using  The  Observer  ®  (2003)  software. 

First,  the  stage  of  the  planning  process  that  the  team  was  performing  at  the  moment  was 
noted.  The  stages  considered  here  were:  understanding  the  mission,  understanding  the 
present  situation,  identifying  possible  courses  of  actions  (COAs),  evaluating  suggested 
COA(s),  and  deciding  on  a  CO  A.  There  was  also  a  category  for  other  activities  (such  as 
having  lunch). 

Each  phase  were  divided  into  subcategories: 

In  understanding  the  mission,  the  subcategories  were:  identifying  criteria  for  success,  i.e.,  the 
factors  that  would  lead  to  success  in  accomplishing  their  mission,  and  efforts  to  understand 
the  mission.  There  should  also  have  been  a  subcategory  for  formulating  a  preliminary  goal 
vision  (end  state).  Now,  this  had  to  be  treated  in  the  notes  (described  below). 

In  understanding  the  present  situation  phase,  the  subcategories  were:  own  resources,  enemy 
resources,  situation  for  civilians,  terrain,  weather,  and  visibility  conditions,  and  a  category 
for  other  considerations  concerning  the  situation.  Some  teams  discussed  third  parties,  for 
example. 

For  identifying  possible  courses  of  action,  the  subcategories  were:  own  COAs,  and  enemy 
COAs.  Notes  (see  below)  were  added  to  describe  if  the  most  likely  and  dangerous  enemy 
COAs  were  identified,  and  if  ways  to  handle  them  were  considered  in  any  detail. 

For  evaluating  suggested  COA(s)  it  was  noted  whether  it  was  done  by  reasoning,  or  by  war 
gaming. 

Deciding  on  a  CO  A  was  treated  as  a  broader  category  containing:  deciding  on  a  CO  A, 
deciding  on  a  concept  of  operation,  and  writing  an  order. 
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For  the  entire  session,  it  was  noted  who  was  doing  what  with  whom.  Each  team  member  was 
assigned  a  code,  but  the  group  was  also  treated  as  a  unit  when  the  whole  team  was  involved 
in  the  same  task.  For  all  team  members,  it  was  noted  in  which  of  the  abovementioned 
category  he  or  she  was  working,  and  for  how  long,  as  well  as  how. 

The  “how”  part  consisted  of  one  of  the  following  alternatives:  reporting  (giving  a  more 
formal  report  on  his  or  her  work),  saying  (one  person  says  something),  discussing  (more  than 
one  person  involved),  quiet  (says  nothing),  leading  (performing  leadership  activities,  such  as 
organizing  staff  work,  giving  orders,  etc.),  in  concert  (all  team  or  a  subgroup),  alone,  together 
with  (someone),  reading  (the  order  for  information),  and  if  they  leave  or  return,  and  as  a 
group  (if  they  had  done  something  on  their  own  and  now  accompanied  the  rest  of  the  group 
in  what  they  were  presently  occupied  with). 

For  each  entry,  it  was  possible  to  add  a  short  note,  if  necessary.  More  extensive  comments 
were  recorded  in  side  notes. 

Observer  Judgments  of  Sensemaking  Performance 

From  the  output  of  the  coding  procedure,  together  with  her  overall  impression  from  observing 
the  teams  working,  the  first  author  graded  each  phase  of  the  team’s  sensemaking  process:  1) 
Understanding  the  mission,  2)  Understanding  the  present  situation,  3)  Identifying  possible 
COAs,  and  4)  Evaluating  suggested  COAs.  5)  The  generation  of  criteria  for  success  was 
treated  as  a  separate  part,  because  it  was  an  activity  that  was  extended  over  the  phases.  6)  The 
observer  estimated  the  degree  of  (common)  sense  arrived  at  by  the  team  as  a  whole.  7) 
Finally  the  observer  graded  the  leadership  performance  of  the  commander,  in  terms  of 
hindering  or  facilitating  the  team’s  sensemaking  process.  The  grades  were  given  on  a  six-step 
Likert-scale,  where  1  was  very  bad  and  6  very  good.  This  means  that  the  teams  could  achieve 
42  points  at  most,  and  7  points  at  least. 

The  process  of  understanding  the  mission,  quite  naturally  starts  with  some  time  during 
which  the  team  members  read  the  order  quietly.  It  was  noted  if  they  appeared  to  spend 
sufficient  time  reading,  and  if  they,  after  reading,  started  working  or  if  they  spent  some  time 
leisurely  chatting.  Most  of  this  period,  the  commander  is  away  at  the  briefing  with  his 
superior  commander.  The  extent  to  which  the  team  commenced  working  in  absence  of  the 
commander  was  noted  (initiative  considered  good),  if  they  worked  in  an  organized  manner, 
and  if  they  discussed  the  mission  and  made  efforts  to  collaboratively  understand  their  task. 
Throughout  the  entire  process,  information  sharing,  involving  all  team  members,  was 
regarded  as  desirable. 

For  the  process  of  understanding  the  present  situation,  how  the  different  tasks  were 
divided  among  the  team  members,  was  registered.  For  an  individual  to  contribute 
substantially  to  the  sensemaking  process,  and  to  arrive  at  a  good  sense  for  him  or  herself,  it  is, 
in  all  likelihood,  important  to  have  an  area  of  responsibility  that  contributes  to  the  solution  of 
the  task  at  hand.  This  would  give  the  individual  something  to  which  he  or  she  can  relate  the 
information  acquired.  It  would  probably  also  motivate  a  greater  expenditure  of  effort  by  the 
team  member  than  if  his  or  her  role  were  felt  to  be  redundant.  Thus,  it  would  both  equip  the 
team  member  with  a  perspective  for  organizing  his  or  her  sensemaking,  and  with  the 
incentive  to  put  some  effort  into  it.  The  major  focus  was  on  the  staffs  report  of  their  work. 
When  did  it  take  place?  Had  the  team  members  been  allowed  sufficient  time  to  penetrate  their 
different  areas  of  responsibility?  Did  they  cover  all  relevant  areas,  did  they  give  clear  and 
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detailed  reports,  and  what  was  the  response  of  the  other  team  members?  Did  they  react  with 
questions  and  discussion,  or  were  they  quiet  receivers?  The  teams  received  high  grades  if 
they  appeared  thorough  in  their  work,  and  if  there  was  a  high  degree  of  exchange  of 
information  and  ideas. 

The  phase  of  identifying  possible  COAs  was  graded  accorded  to  criteria  similar  to  those 
described  for  understanding  the  situation.  It  was  noted  how  labour  was  divided  in  the  process 
of  generating  own  and  enemy  COAs,  how  those  COAs  were  reported,  how  many  COAs  were 
generated,  if  the  most  likely  and  the  most  dangerous  enemy  COAs  were  identified,  and  if 
ways  of  dealing  with  them  were  discussed  in  any  detail.  Some  teams  also  considered  possible 
COAs  for  the  civilians,  which  increased  their  scores.  The  more  of  the  team  members  who  had 
important  roles  in  this  process,  and  the  more  each  team  member  contributed  to  a  discussion, 
the  better  the  sensemaking  process  was  considered  to  be. 

In  evaluating  suggested  COA(s),  two  factors  were  appraised.  First,  the  process  of  comparing 
the  different  alternatives  suggested,  if  more  than  one,  and  putting  the  one  chosen  for  serious 
consideration  under  scrutiny,  looking  for  weaknesses,  and  discussing  how  to  deal  with  these. 
The  more  thorough  this  work  the  better,  and  the  more  team  members  engaged  in  the 
discussion  the  better. 

Second,  there  was  the  question  whether  they  put  the  preliminary  plan  to  test  in  war  gaming  or 
not.  It  was  considered  better  if  they  did  than  if  they  did  not.  War  gaming  makes  it  explicit  to 
all  team  members  what  it  is  that  the  team  has  agreed  upon  (or  the  commander  has  decided  to 
put  to  test). 

The  generation  of  criteria  for  success  was  a  process  continuing  more  or  less  from  start  to 
the  arrival  at  a  concept  of  operation.  This  made  it  difficult  to  follow,  and  the  score  was  only 
based  on  to  what  extent  criteria  of  success  were  discussed  in  any  organized  fashion  in  the 
teams,  and  the  extent  to  which  all  team  members  were  involved  in  these  discussions. 

Team  sense  was  an  estimate,  based  on  the  impression  from  the  entire  process,  of  how  clear, 
and  complete,  was  the  (common)  sense  achieved  by  the  teams.  The  observer  also  noted  how 
well  developed  was  the  basis  for  the  chosen  COA,  as  well  as  how  elaborate  was  this  COA, 
and,  above  all,  how  well  all  members  of  the  team  knew  all  this. 

The  commander  performance  score  was  based  on  how  well  the  commander  (or  commander 
together  with  his  chief  of  staff)  organized  the  staff  work.  Did  he  (they  were  all  male)  divide 
the  work  among  the  team  members  in  accordance  with  what  has  been  hypothesized  to 
facilitate  sensemaking?  Did  all  team  members  have  tasks  important  to  the  sensemaking 
process,  to  understanding  the  situation  and  to  generate  a  suitable  course  of  action?  Did  he 
manage  to  keep  everybody  busy  and  involved  in  work?  Did  he  keep  track  of  all  steps  in  the 
process,  seeing  to  that  nothing  important  was  forgotten?  Did  he  encourage  discussion,  and 
did  he  summarize  the  discussions  to  make  sure  that  everybody  was  on  the  same  track  before 
continuing  to  the  next  step  in  the  process?  And,  did  he  strive  to  maintain  a  productive  social 
atmosphere  in  the  team? 

In  the  low  uncertainty  teams,  it  was  noted  how  they  made  use  of  the  provided  additional 
information. 

Selection  of  High  and  Low  Performing  Teams  According  to  Expert  Judgments 
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As  noted  above,  the  two  most  successful  and  the  two  least  successful  in  each  condition  (low 
uncertainty  and  normal  uncertainty),  were  selected  for  analysis,  in  total  eight  teams, 
according  to  the  mean  ratings  of  the  produced  plans  by  two  military  expert.  (For  details 
concerning  the  criteria  used  by  the  raters  we  refer  to  Thunholm,  2005).  The  inter-rater 
agreement  was  moderate  (r  =  0.64).  One  of  the  high  performing  teams  in  the  normal 
uncertainty  condition  had  to  be  exchanged  for  the  next  best  to  it  in  the  same  condition.  The 
video  camera  recording  that  team  ceased  to  function  in  mid-session,  so  we  were  lacking 
information  on  the  later  stages  of  their  sensemaking  process. 

The  person  rating  the  sensemaking  process,  the  first  author  of  this  paper,  was  blind  to  the 
evaluation  of  the  team’s  plans,  and  she  was  in  no  position  to  assess  the  cleverness  of  the  ideas 
put  forth,  due  to  lack  of  military  education.  She  was  only  able  to  assess  the  way  the  teams 
were  working  and  interacting. 


Results  and  Discussion 

We  identified  a  sensemaking  process  made  out  of  two  parts  or  processes:  an  initial  process 
when  the  team  strives  to  understand  the  mission  and  identify  what  aspects  of  the  situation  is 
relevant  to  it,  finding  out  what  is  known  and  unknown,  what  is  possible  to  find  out  and  what 
is  not,  what  could  possibly  be  accomplished  and  what  the  enemy  might  be  able  to  carry  out. 
This  phase  is  most  clearly  independent  of  planning;  it  occurs  at  least  some  time  before  the 
planning  may  begin.  Some  basic  sense  is  needed  on  which  to  base  the  planning. 

In  the  second  process,  the  team  works  to  figure  out  what  would  be  best  to  do,  what  might 
possibly  happen,  and  how  to  deal  with  every  conceivable  turn  of  events.  This  is  based  on 
what  has  been  accomplished  in  the  first-mentioned  process.  This  process  is  also  part  of  the 
planning  process;  considering  possible  courses  of  action  is  an  initial  step  in  the  generation  of 
a  plan  and  it  shows  that  there  is  some  overlap  between  sensemaking  and  planning.  The 
processes  are  not  completely  sequential;  even  though  the  first  is  the  one  first  to  be  initiated, 
followed  by  the  second,  they  overlap  substantially.  There  are  shifts  back  and  forth  between 
them,  but  they  are,  anyhow,  processes  dealing  with  different  aspects  of  military  decision 
making. 

The  second  process  is  followed  by  a  third,  which  belongs  more  or  less  exclusively  to  the 
planning  process,  namely  the  process  of  working  out  the  finer  details  of  the  COA  decided 
upon  and  transforming  it  into  orders,  or  missions,  to  subordinate  units.  This  process  was 
excluded  from  the  analysis  in  the  present  study,  since  it  is  not  a  part  of  the  sensemaking 
process. 

The  ratings  of  the  sensemaking  process  of  the  selected  teams  are  summarized  below  in  Table 
1.  In  the  Cond  column,  it  is  noted  which  condition  the  teams  worked  in:  low  uncertainty 
(Low)  or  normal  uncertainty  (Norm).  In  the  Expert  column,  it  is  noted  which  teams  were  the 
high  and  the  low  performers  according  to  the  expert  ratings  of  their  plans.  The  N  column 
gives  the  number  of  team  members,  UM  stand  for  understanding  the  mission,  UPS  for 
understanding  the  present  situation,  IPCoA  for  identifying  possible  COAs,  ECoA  for 
evaluating  suggested  COA(s),  CrS  for  generation  of  criteria  for  success,  TS  for  team 
sense,  and  Com  for  commander  performance.  Sum  is  the  sum  of  the  ratings. 
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Team 

Cond 

Expert 

N 

UM 

UPS 

IPCoA 

ECoA 

CrS 

TS 

Com 

Sum 

1 

Norm 

High 

6 

4 

4 

4 

5 

5 

6 

5 

33 

2 

Low 

High 

7 

4 

6 

5 

6 

5 

6 

6 

38 

3 

Norm 

Low 

6 

3 

2 

2 

1 

3 

2 

2 

15 

4 

Low 

Low 

6 

4 

3 

4 

1 

3 

2 

2 

19 

5 

Norm 

Low 

7 

3 

4 

3 

6 

4 

5 

4 

29 

6 

Low 

Low 

6 

3 

2 

2 

4 

3 

4 

3 

21 

7 

Low 

High 

6 

3 

4 

5 

4 

5 

6 

5 

32 

8 

Norm 

High 

7 

4 

6 

5 

4 

6 

6 

6 

37 

Table  1.  Results  From  the  Ratings  of  the  Sensemaking  Processes. 

For  each  measure,  t-tests  were  performed  to  assess  the  effects  of  uncertainty  and  planning 
performance.  For  uncertainty,  there  were  no  significant  effects  (p  >  .05).  For  planning 
performance,  the  teams  ranked  high  performed  significantly  (p  <  .05)  better  for  understanding 
present  situation  (5.00  vs.  2,75,  t6  =  3,00),  identifying  possible  courses  of  action  (4.75  vs. 
2,75,  t6  =  3.70),  generating  criteria  for  success  (5.25  vs.  3.25,  t6  =  5.65),  team  sense  (6.00  vs. 
3.25,  t6  =  3.67),  commander  performance  (5.50  vs.  2.75,  t6  =  4.92)  and  the  sum  of  all  ratings 
(35.0  vs.  21.0,  t6  =  4.25),  but  not  for  understanding  the  mission  and  evaluation  of  COAs.  The 
most  important  variables  here  are:  understanding  mission,  understanding  present  situation, 
and  identifying  possible  COAs,  which  belong  to  the  first  sub-process  described  above,  and 
provide  indices  of  sensemaking  that  are  fairly  independent  of  the  planning  process.  For  all 
these  indices  except  understanding  the  mission,  we  have  significant  effects,  and  for 
understanding  the  mission,  the  difference  between  teams  ranked  high  and  low  is  the  right 
direction  (5.0  vs.  3.25).  These  findings  suggest  that  sensemaking  is  an  important  factor  that 
determines  the  quality  of  the  work  produced  in  a  military  staff  unit.  Moreover,  the  fact  that 
the  index  of  commander  performance  was  significantly  higher  for  the  teams  producing  better 
plans  suggests  that  the  commander’s  behavior  is  an  important  factor  in  facilitating  the 
sensemaking  process.  Most  important,  the  results  demonstrates  that  it  is  possible  to  measure 
sensemaking,  even  using  the  fairly  rough  measures  reported  on  here,  and  to  measure  it 
independently  of  planning. 

It  was  difficult  to  rate  the  quality  of  the  understanding  the  mission  phase.  This  is  because 
we  did  not  observe  the  sensemaking  process  with  respect  to  understanding  the  mission 
directly.  We  could  only  infer  their  understanding  of  the  mission  indirectly.  As  explained 
above,  the  teams  were  allowed  scheduled  time  (two  hours)  to  study.  In  addition,  they  had  45 
minutes  to  prepare  before  they  received  the  division  order.  This  period  of  time  was  not 
registered  on  videotape,  and  therefore  we  do  not  know  what  the  teams  were  doing  during  that 
time.  This  means  that  information,  relevant  to  the  understanding  of  the  mission,  and  how  the 
teams  organized  their  work  during  this  phase,  was  lacking. 

The  most  critical  phase  seems  to  be  the  understanding  the  present  situation  stage,  or  the 
period  following  the  commander’s  return  from  the  order  briefing.  Here  the  teams  receive 
ratings  ranging  from  2  to  6,  and  they  tend  to  receive  similar  ratings  on  the  remaining  criteria. 
It  seems  important  that  all  team  members  are  involved  in  the  process  of  making  sense  of  the 
situation,  and  that  they  frequently  and  thoroughly  inform  each  other  of  their  progress.  The 
exchange  of  thought  and  information  appears  to  be  more  important  than  to  avoid  disturbing 
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each  other’s  thinking  processes.  Reports  ought  to  be  rather  detailed  and  elaborate  and  be 
followed  by  questions  and  discussion. 

The  effects  tend  to  accumulate  over  time.  The  differences  between  the  team’s  sensemaking 
processes  become  increasingly  pronounced.  It  seems  to  be  difficult  to  make  up  later  for  an 
unsuccessful  start. 

In  the  phase  of  identifying  possible  (own  and  enemy)  COAs  too,  the  involvement  of  all  team 
members  is  important.  There  might  be  a  division  of  responsibility  for  the  own  and  the  enemy 
perspective,  but  all  members  should  be  involved  in  a  discussion  of  possible  alternatives.  The 
number  of  alternatives  considered  seems  less  important  than  the  thoroughness  of  their 
treatment.  Some  teams  appeared  to  produce  alternatives  simply  for  the  sake  of  it,  which  is 
probably  not  of  much  use. 

When  evaluating  suggested  COA(s),  which  belongs  to  the  second  sub-process  of 
sensemaking  mentioned  above,  and  is  also  part  of  the  planning  process,  it  is  important  to 
attack  the  alternative  under  consideration  from  every  possible  angle.  To  do  this  successfully, 
it  is  important  that  all  team  members  take  part  in  the  discussion.  War  gaming  might  be 
helpful.  It  put  the  preliminary  plan  to  a  stage-wise  test,  highlighting  the  time  dimension, 
making  is  easier  to  consider  logistic  issues,  and  identify  important  details  that  have  to  be  dealt 
with.  War  gaming  is  probably  more  useful  as  a  means  to  put  the  plan  to  test  and  working  out 
the  details,  than  as  a  confirmatory  playing  through  of  an  already  agreed-upon  COA. 

As  already  mentioned  in  the  Method  section,  the  generation  of  criteria  for  success  was 
difficult  to  rate.  They  were  generally  listed  on  paper,  not  possible  to  read  from  the  video 
recordings,  and  they  were  unfortunately  not  collected  and  saved  after  the  exercise.  This  made 
it  difficult  to  follow  the  process,  and  it  was  only  the  amount  of  time  spent  explicitly 
discussing  them  that  could  be  assessed.  This  is  clearly  an  important  aspect  of  the 
sensemaking  process  since  it  concerns  aspects  of  what  can  be  done  to  achieve  the  mission.  It 
is  therefore  important  that  a  significant  difference  between  teams  whose  plans  were  ranked 
high  and  teams  whose  plans  were  ranked  low  nevertheless  emerged. 

The  team  sense,  an  estimate  of  the  degree  of  (common)  sense  arrived  at  by  the  teams,  which 
could  only  be  inferred  rather  than  measured  directly,  is  important  for  two  reasons.  First,  for 
the  collectively  written  order  to  be  clear  and  coherent,  it  is  essential  that  all  its  contributing 
writers  have  as  similar  a  mission  as  possible  in  mind.  If  not,  they  will  write  parts  of  different 
orders,  something  that  is  certain  to  confuse  the  subordinates.  Second,  when  the  plan  is  to  be 
implemented,  it  is  vital  that  all  are  striving  towards  the  same  goal,  by  means  agreed  upon. 

All  commanders  conveyed  more  or  less  the  same  information  from  the  order  briefing.  It  was 
probably  less  important  what  the  commander  said,  because  all  information  delivered  was  also 
part  of  the  written  order,  than  how  he  said  it.  That  the  commander  approaches  his  task 
ambitiously  and  seriously,  and  thus  inspires  the  team  members  to  do  the  same,  may  possibly 
be  of  great  importance.  It  may,  of  course,  work  in  the  opposite  direction  as  well.  A  hard 
working  team  may  encourage  a  commander  to  give  his  best  effort  to  his  task. 

One  major  task  for  the  commander  (regardless  of  whether  one  or  two  individuals  served  that 
function;  some  teams  used  a  chief  of  staff  as  well)  is  to  make  sure  that  all  staff  members  work 
on  important  tasks.  In  addition,  he  (or  she)  must  supervise  the  process  as  a  whole, 
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summarizing  the  work  to  make  sure  all  members  agree  upon  the  interpretations  made, 
encouraging  discussion,  making  sure  everybody  have  his  or  her  say  in  the  matter  discussed. 

For  one  of  the  low  performing  teams,  team  5,  it  was  somewhat  difficult  to  predict  the  quality 
of  the  resulting  plan.  The  commander  was  in  strong  control  of  his  staff  members.  He  used  the 
staff  members  to  serve  his  own  sensemaking  process.  From  a  team  sensemaking  perspective 
this  is  bad,  but  if  the  commander  is  a  very  competent  individual  and  the  task  not  too  complex 
for  one  person  to  handle,  then  the  resulting  plan  might  still  be  quite  good.  This  would  require 
the  commander  to  supervise  every  step  in  the  procedure,  which  he,  in  fact,  did.  In  this  case, 
the  resulting  plan  was  of  low  quality,  which  lends  support  to  the  importance  of  a  sensemaking 
process  including  all  team  members,  or  at  least  as  many  as  possible.  The  team  spent  40 
minutes  war  gaming  and  during  that  phase  they  were  discussing  the  COA  under 
consideration.  This  led  the  rater  to  infer  that  the  team  sense  might  in  fact  be  quite  good.  This 
seems,  however,  to  have  been  an  unwarranted  assumption. 

The  level  of  uncertainty  neither  affected  the  sensemaking  process  nor  the  quality  of  the  plans. 
The  only  thing  observed  was  that  the  teams  in  the  low  uncertainty  conditions  warned  each 
other  of  getting  stuck  in  details.  The  reason  for  the  lack  of  effect  could  be  that  the  teams  were 
unable  to  make  good  use  of  the  detailed  information  on  the  enemy,  that  they  had  more 
pressing  issues  to  attend  to,  or  that  the  participating  officers  were  so  good  at  planning  under 
normal  conditions  that  a  ceiling  effect  was  achieved.  The  latter  explanation  is  rather  unlikely, 
however.  There  was  substantial  variance  in  performance  in  both  conditions,  and  the  overall 
impression  by  the  expert  raters  was  that  there  was  ample  room  for  improvements  for  most 
teams. 


Conclusions 

That  sensemaking  performance  could  be  measured  and  explain  the  quality  of  plans  resulting 
from  the  process,  are  encouraging  results  indeed.  It  suggests  that  it  will  be  possible  to 
perform  experiments  testing  the  effect  of  different  technical  solutions  and  work 
organizations,  for  example,  on  the  sensemaking  process. 

Superior  information  failed  to  make  the  teams  produce  superior  plans.  The  fog  of  war  may 
not  reside  in  the  environment  only,  it  may  be  found  inside  the  heads  of  the  decision  makers  as 
well.  Having  information  alone  is  not  sufficient;  it  has  to  make  sense  as  well. 

The  study  reported  on  here  was  a  pilot  study,  and  the  measures  used  were  based  on  the 
subjective  ratings  of  one  researcher.  The  results  from  these  efforts  were,  however,  promising. 
A  natural  next  step  will  be  to  produce  clear  descriptions  of  the  rating  criteria,  and  have 
different  raters  code  and  grade  the  same  material  to  test  the  inter-rater  reliability.  If  it  is 
sufficiently  high,  the  rating  procedure  can  be  used  for  experimental  purposes  as  a  quality 
measure  of  the  sensemaking  procedure. 

The  students  participating  in  the  planning  exercise  studied  all  had  the  same  military  rank  and 
fairly  similar  backgrounds.  They  served  specialized  roles  in  the  teams,  but  they  did  not  have  a 
specialist’s  competence  for  their  role.  The  impact  of  professional  expertise  and  of  having 
members  with  different  professional  backgrounds,  on  the  sensemaking  process,  needs  to  be 
addressed  by  subsequent  investigations. 
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Sensemaking  in  Command  and  Control 

is  action  oriented 

It  is  focused  on  the  accomplishment  of  an 
assigned  mission. 

The  purpose  is  to  arrive  at  an  understanding  of 
the  situation  in  terms  of  what  should  be  done, 
and  not  merely  what  is  happening.  This  affects 
how  the  task  is  approached. 

In  real  combat  there  is  no  way  to  know  if  the 
situation  is  correctly  understood,  but  one  will 
notice  if  actions  taken  are  followed  by  the  desired 
consequences. 


Purpose  of  Research 


to  find  a  suitable  model  for  describing 
sensemaking  in  a  military  command  team,  and 
the  factors  that  affect  this  process 

to  find  a  method  to  measure  the  quality  of  the 
sensemaking  process  at  its  different  stages, 
and  the  products  resulting  from  it 

in  order  to  be  able  to  test  in  controlled 
experiments  the  effect  of  various  efforts  to 
improve  military  sensemaking,  and  thereby 
also  military  planning 


Sensemaking  in  C2 


Each  individual  identifies  information  relevant  to  his  or  her 
professional  role  in  the  present  situation,  based  on  the  mission, 
rules  of  engagement,  and  his  or  her  knowledge  base  and  personal 
values,  and  creates  his  or  her  individual  sense. 

Each  individual  creates  his  or  her  own  sense,  and  by  interacting 
with  each  other,  communicating  and  exchanging  thought  a,  to  some 
extent,  shared  sense  emerges. 

The  commander  is  the  person  who  ultimately  decides  on  the  course 
of  action,  and  the  commander  sense  is  ,  for  this  and  other  reasons, 
treated  separately. 


Important  Factors 


Shared  knowledge:  The  more  the  team  members  have  to  explain  to 
each  other  the  slower  and  more  cumbersome  the  process. 

Social  climate  and  interaction:  The  more  information  and  thoughts 
are  shared,  constructive  ideas  as  well  as  critique,  the  better. 

Organization  of  work:  Well-organized  work  where  good  use  is 
made  of  all  team  members  is  desirable.  The  role  taken  by  the 
commander  is  considered  an  important  aspect. 


Method  (I) 


Participants:  16  Teams  of  5-7  Army  captains,  one  team 
member  acting  as  commander 

Scenario:  A  battle  scenario  at  the  tactical  (brigade)  level,  in 
the  area  of  Stockholm,  the  capital  of  Sweden. 

Task:  To  produce  parts  of  a  brigade  order. 

Time:  6  hours. 


All  teams  were  video  recorded. 


Method  (II) 


Experimental  Design: 

Two  Conditions:  8  teams  NCW  Info  —  Low  Uncertainty 

8  teams  —  Normal  Uncertainty 

Expert  ratings  of  plan  quality:  Two  independent  raters 
(inter-rater  agreement  r  —  0.64) 

Sensemaking  ratings:  The  teams  producing  two  best  and 
two  worst  plans  in  each  condition,  in  total  eight  teams, 
were  selected,  and  their  sensemaking-process  were  rated 
blindly. 


Analysis  (I) 


Coding  of  Videos: 

The  actions  of  the  team  members  were  observed: 

■  which  stage  of  the  planning  process  the  teams  were 
performing:  understanding  the  mission,  understanding 
the  present  situation,  identifying  possible  courses  of 
actions  (COAs),  evaluating  suggested  COAs,  and 
deciding  on  a  COA,  and 

■  how  the  team  members  worked.  If  they  were  working 
by  themselves,  together  with  someone,  discussing  all 
team  together,  giving  more  formal  reports,  leading  and 
organizing  the  work,  and  so  on. 


Analysis  (II) 


Sensemaking  Performance: 

On  basis  of  the  observations,  the  teams’  sensemaking 
performance  in  the  stages  understanding  the  mission, 
understanding  the  present  situation,  identifying 
possible  COAs,  and  evaluating  suggested  COA(s)  was 
graded,  and  so  was  generation  of  criteria  for  success,  an 
estimate  of  the  team  sense,  and  commander  behavior. 

The  grades  were  made  on  a  six-step  scale,  where  1  was  very 
bad  and  6  very  good.  (Max  =  42,  Min  =  7) 


Results  (I) 


Expert  Rating 

Uncertainty 

Sensemaking 

Good 

Low 

38 

Good 

Low 

32 

Good 

Normal 

33 

Good 

Normal 

37 

Bad 

Low 

19 

Bad 

Low 

21 

Bad 

Normal 

15 

Bad 

Normal 

29 

Results  (II) 

Good  Teams  Make  Good  Sense 


The  teams  producing  the  best  plans  were  also  the  teams 
with  the  best  sensemaking  process,  mean  =  35,  compared 
to  a  mean  of  21  for  the  teams  producing  the  worst  plans. 
t6  =  4.24 

There  was  no  difference  in  performance,  neither  in  plan 
quality,  nor  in  sensemaking  between  the  experimental, 
uncertainty  conditions. 


Conclusions 


Sensemaking  in  C2  consists  of  two  processes: 

■  understanding  the  mission  itself,  the  task  at  hand 

■  finding  out  how  to  best  solve  it  under  the  present 
conditions 

The  first  is  more  independent  of  planning,  the  second  is 
also  part  of  the  planning  process. 

Sensemaking  central  and  important  activity  in 
command  and  control.  A  good  sensemaking  process  is  a 
precondition  for  successful  planning  and  subsequent 
good  orders/missions  to  subordinates. 


Questions  or  Comments? 


